Apparatus and method for reducing quantization error in digital image signals

ABSTRACT

In an image signal processing apparatus and processing method, intermediate gradation is produced by way of a bit expansion to reduce a quantizing noise. The image signal processing apparatus is comprised of: bit expanding means for bit-expanding an n-bit quantized input image signal into an m-bit (symbols “n” and “m” are integers, and own a relationship of n&lt;m); control signal output means for outputting a control signal based upon the input image signal; and a converting unit for adaptively converting the signal from the bit expanding means into an m-bit signal in response to the control signal from the control signal output means. Also, the image signal processing method comprises the steps of: bit-expanding an n-bit quantized input image signal to an m-bit (symbol “n” and “m” are integers and own a relationship of n&gt;m); and adaptively producing lower order (m−n) bits in accordance with an image nature of the input image signal to thereby be converted into an m-bit signal.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention generally relates to an image signalprocessing apparatus, an image signal processing method, and an imagesignal decoding apparatus. More specifically, the present invention indirected to an image signal processing apparatus, an image signaldecoding apparatus capable of reducing a quantizing error when anencoded image signal is decoded to be displayed.

[0003] 2. Description of the Related Art

[0004] In such a system, a moving picture signal is recorded on arecording medium such as a magneto optical-disk and a magnetic tape,this moving picture signal is reproduced so as to be displayed. Anmoving picture signal is transmitted from a transmitter via atransmission path to a receiver, as in a TV conference system, a TVtelephone system, and a broadcasting appliance. In these systems, sincethe transmission path and the recording medium are utilized at a highefficiency, the image signals are compressed and coded by utilizing theline correlation as well as the frame correlation of the video signals.

[0005] A description will now be made of the high-efficiency coding ofthe moving picture signal.

[0006] Conventionally, moving image data such as video signals containsvery large quantities of information. Thus, to record/reproduce thismoving image data for a long time duration, the recording mediums whosedata transfer speeds are very high are necessarily required. As aresult, a large-sized magnetic tape and a large-sized optical disk areneeded. When moving image data is transmitted via the transmission path,or broadcasted, since an excessive data amount must be transferred, suchmoving image data could not be directly transmitted via the presentlyexisting transmission path.

[0007] Under such a circumstance, when a video signal is recorded on acompact recording medium for a long time, or is used in a communicationand a broadcasting system, the video signal must be processed in thehigh-efficiency coding for a recording purpose, and further such a meansfor decoding this read signal at a higher efficiency must be employed.To accept such needs, the high-efficiency coding methods by utilizingthe video signal correlation have been proposed. As one of these method,so-called “MPEG (moving picture experts group) 1 and 2” have beenproposed. This method has been proposed as a standard method inISO-IEC/JTC1/SC2/WG11 conference. That is, such a hybrid systemcombining the movement compensation prediction coding with DCT (discretecosine transform) coding has been employed.

[0008] The movement compensation prediction coding corresponds to themethod for using correlation of the image signal along the time basedirection. The presently entered image is predicted from the signalalready being decoded and reproduced, and only the prediction error istransmitted, so that the information amount required in the data codingis compressed.

[0009] The DCT coding is such a coding method that signal power isconcentrated to a specific frequency component by utilizing the frametwo-dimensional correlation owned by the image signal, and only theconcentrated/distributed coefficients are coded. As a result, theinformation amount can be compressed. For instance, the DCT coefficientat a flat picture portion of which self-correlation is high isconcentrated and distributed into the low frequency component. Thus, inthis case, only the coefficient concentrated/distributed to the lowfrequency component is coded, so that the information amount can bereduced.

[0010] It should be noted that although the MPEG 2 system is employed asthe coding device in the specification, many other coding systems exceptfor the MPEG system may be applied.

[0011] In general, to improve an image impression in a TV monitor, afilter for emphasizing the image is employed. There are various sorts ofemphasizing process filters. For example, there is such a high-passfilter for emphasizing a-high frequency component of an image signal.Also, a so-called “contrast filter” is provided to amplify an amplitudeof an image signal. There is another filter for converting densitygradation.

[0012] These emphasizing filters not only emphasize the image signal toincrease the visual impression, but also the noise contained in thesignal. As a consequence, when the image signal contains many noises,these noises would become apparent, resulting in deterioration of visualimpressions.

[0013] The output signal to the display unit of the TV monitor, andthus, the above-explained emphasizing process is performed after thedigital signal is converted into the analog signal. This is shown inFIG. 1 and FIG. 2.

[0014]FIG. 1 shows such a case that an input to a TV monitor 400 is adigital signal. The digital input signal is D/A-converted into an analogsignal by a D/A converter 401, and then is emphasized by an emphasizingfilter 402. Thereafter, the emphasized signal is supplied to a displayunit 403 for representation.

[0015]FIG. 2 shows another case that an input to a TV monitor 410 is ananalog signal. The analog input signal is emphasized by an emphasizingfilter 412 to be supplied to a display unit 413 for representation.

[0016] In the case of the analog image signal, random noises such aswhite noise are major noises. The noises of the digital signal are blockdistortion, and quantizing noises near edges. These noises are locallyproduced and own higher correlation thereof. When the noise contained inthe digital image signal is emphasized, the visual impression would begreatly deteriorated, and would give unnatural impression.

[0017] Normally, a digital image signal is quantized by 8 bits. In thenormal image signal, quantizing noise could not be visually recognized.In other words, no discrimination can be made of a 1-bit interval in 8bits. However, as shown in FIG. 3, when an image signal is simplyincreased in a flat manner, this quantizing error, namely 1-bit intervalcan be recognized. This is because a human observation is very sensibleto this flat portion, and steps of 1-bit interval are continuous.

[0018] A similar phenomenon appears as to an image with a better S/Nratio. When an image owns a low S/N ratio, the 1-bit interval is mixedwith noises, which cannot be therefore observed. However, as to an imagehaving low noise, the quantizing error (1-bit interval) can bediscriminated. This phenomenon especially occurs in the noise-eliminatedimage, and the signal and CG produced by the image signal generatingapparatus.

[0019] When the image signal is coded, a similar phenomenon occurs. Ageneral image signal contains noise. When the coding bit rate is high,this noise component is also coded to be transmitted. When the codingbite rate is low, this noise component could not be transmitted. At thistime, in the MPEG coding system to perform the block processing, thismay be observed as a block-shaped noise. If such a block-shapeddistortion is continued, even when this corresponds to a step of 1-bitdifference, it could be visually recognized. Since this is observed as apseudo contour, it is called as a “pseudo contour”.

[0020] FIGS. 4A and FIG. 4B represent such a case that pseudo contoursare produced. FIG. 4A indicates a two-dimensional pattern displayed on ascreen. FIG. 4B represents a signal level on a line a to “a” of FIG. 4A.

[0021] A similar phenomenon will occur when a decoded image signal isemphasized. When the image signal is emphasized, a 1-bit differencewould be widened. This phenomenon will now be explained with referenceto FIG. 5A and FIG. 5B.

[0022]FIG. 5A shows a case that a step of a 1-bit difference isconverted into an analog signal. When this signal is emphasized, asshown in FIG. 5B, the 1-bit difference would be widened. As a result,this 1-bit difference could be visually recognized. This phenomenon isvisually recognized as a pseudo contour.

[0023] When the image signal is decoded in the above-described manner,the quantizing error can be visually recognized to produce the pseudocontour.

[0024] Also, there is another possibility that the quantizing error canbe discriminated also in the not-coded image signal. This may be-causedby the performance limits by the 8-bit quantizing process.

[0025] As described above, when the digitally compressed image isobserved on the TV monitor with the emphasizing process, the noise(deterioration) caused by the compression would be emphasized todeteriorate the image impressions. Thus, there is a problem ofoccurrences of unnatural images.

SUMMARY OF THE INVENTION

[0026] The present invention has been made to solve such variousproblems, and therefore, has an object to provide an image signalprocessing apparatus, an image signal processing method, and an imagesignal decoding apparatus, which are capable of suppressing a quantizingnoise even in a coded image signal.

[0027] Another object of the present invention is to provide such asystem that even when a digital image signal is observed by a TV monitorwith a function of a signal emphasizing process operation, a naturallygood image could be reproduced.

[0028] A further object of the present invention is to provide such asystem that even in an original image signal which is coded, aquantizing error caused by a limitation in an 8-bit quantizing processcould not be apparently observed.

[0029] To solve the above-described problems, an image signal processingapparatus, according to the present invention, is featured bycomprising:

[0030] bit expanding means for bit-expanding an n-bit quantized inputimage signal into an m-bit (symbols “n” and “m” are integers, and own arelationship of n<m);

[0031] control signal output means for outputting a control signal basedupon said input image signal; and

[0032] a converting unit for adaptively converting the signal from saidbit expanding means into an m-bit signal in response to the controlsignal from said control signal output means.

[0033] In the converting unit, the input image signal is smoothed, andat the same time, such a process is performed in order not to lose thehigh frequency component of the input image signal. For example, alow-pass filter is employed to perform the signal smoothing. Tocompensate for the high frequency component, the original input imagesignal is adaptively used, and the n-bit input image signal is convertedinto an m-bit signal.

[0034] This may be similarly applied to such an image signal decodingapparatus capable of decoding such an image signal which has been codedby the prediction image coding method.

[0035] In the bit expanding means, (m−n) bits of “0” are added to an LSB(least significant bit) of an n-bit input image signal in order tosimply perform the bit expansion from n bits into m bits. In the controlsignal output means, the converting unit is adaptively controlled inresponse to the input image signal, so that the high frequency componentof the input image signal is not lost. In the bit converting means, them-bit smoothed signal is outputted within such a range that the highfrequency component is not lost in response to the control signal.

BRIEF DESCRIPTION OF THE DRAWINGS

[0036] For a better understanding of the present invention, reference ismade of a detailed description to be read in conjunction with theaccompanying drawings in which:

[0037]FIG. 1 is a schematic block diagram for representing one exampleof a digital signal input type monitor apparatus;

[0038]FIG. 2 is a schematic block diagram for representing one exampleof an analog signal input type monitor apparatus;

[0039]FIG. 3 illustrates an example of such a flat image signalincreased monotonously;

[0040]FIG. 4A and FIG. 4B are explanatory diagrams for explanatingpseudo contour when a block-shaped distortion is continued;

[0041]FIG. 5A and FIG. 5B are diagrams for showing an image signalemphasized processed;

[0042]FIG. 6 is a schematic block diagram for indicating an arrangementof an image signal processing apparatus according to an embodiment ofthe present invention;

[0043]FIG. 7-is a diagram for explaining such an operation to bit-expand8 bits into 10 bits;

[0044]FIG. 8 is a block circuit diagram for showing a concrete exampleof a tow-dimensional low-pass filter (LPF);

[0045]FIG. 9A shows a concrete example of a 3×3 pixel block;

[0046]FIG. 9B indicate a concrete example of a two-dimensional low-passfilter (LPF);

[0047]FIG. 10A and FIG. 10B are diagrams for showing a comparisonexample between an input image signal and a signal outputted from alow-pass filter;

[0048]FIG. 11A is an illustration for showing 1 bit of an 8-bit signal;

[0049]FIG. 11B is an illustration for representing 1 bit of a 10-bitsignal;

[0050]FIG. 12A and FIG. 12B are diagrams for explaining the principleidea of the high-efficiency coding;

[0051]FIG. 13a and FIG. 13B are diagrams for explaining picture types inthe case that image data is compressed;

[0052]FIG. 14A and FIG. 14B are diagrams for explaining the basic ideato encode a moving picture signal;

[0053]FIG. 15 is a schematic block diagram for indicating a structuralexample of an image signal encoding apparatus and an image signaldecoding apparatus to which the embodiment of the present invention isapplied;

[0054]FIG. 16 is an explanatory diagram for explaining a formatconverting operation of a format converting circuit 17 shown in FIG. 15;

[0055]FIG. 17 is a schematic block diagram for representing a structuralexample of an encoder 18 shown in FIG. 15;

[0056]FIG. 18A and FIG. 18B are diagrams for explaining operation of aprediction mode switching circuit 52 of FIG. 17;

[0057]FIG. 19A and FIG. 19B are diagrams for explaining operation of aDCT mode switching circuit 55 of FIG. 17;

[0058]FIG. 20 is a schematic block diagram for showing a structureexample of a decoder 31 shown in FIG. 15;

[0059]FIG. 21 is a schematic block diagram for representing an imagesignal processing apparatus according to another embodiment of thepresent invention;

[0060]FIG. 22 is a graphic representation for indicating a relationshipbetween a coefficient of a filter strength determining circuit anddispersion of an image signal;

[0061]FIG. 23 is a graphic representation for indicating a relationshipbetween a coefficient of a filter strength determining circuit and adynamic range of an image signal; and

[0062]FIG. 24 is a graphic representation for showing a relationshipbetween a coefficient of a filter strength determining circuit and aluminance signal level of an image signal.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0063] Referring now to drawings, several preferred embodiments of thepresent invention will be described.

[0064] It should be noted that although the following embodiments willdescribe such a preferable example when a moving picture iscompression-coded by utilizing line correlation and frame correlation,the present invention may be similarly applied to other image signalcoding systems. Also, the present invention is not limited to acompressed image signal. Furthermore, although the below-mentionedembodiments will disclose such a case when an 8-bit digital image signalis converted into a 10-bit digital image signal, any bit lengths may beutilized. In general, the present invention may be applied to such acase that n bits are converted into m bits (note that symbols “n” and“m” denote integers, n<m).

[0065]FIG. 6 schematically shows an image signal processing apparatusconstituting a first embodiment of the present invention.

[0066] In the first embodiment shown in this FIG. 6, an n-bit (forexample, 8 bits) image signal S1 is supplied into an input terminal 101.This n-bit (for instance, 8 bits) input image signal is entered into abit expanding means for bit-expanding the n-bit image signal into anm-bit image signal (m>n), for example a 10-bit expanding circuit 102. Inthe 10-bit expanding circuit 102, as shown in, e.g., FIG. 7, 2 bits of“0” are added to an LSB (least significant bit) of the entered 8-bitimage signal so as to perform the bit expansion, i.e., to output theadded signal as a 10-bit image signal S2. In general, (m−n) bits of “0”may be added to a lower sided of an n-bit image signal than an LSBthereof to thereby produce an m-bit image signal.

[0067] The output signal S2 of the 10-bit expanding circuit 102functioning as the bit expanding means is transferred to a controlsignal output means 120 for outputting a control signal based upon animage character of the input image signal, and a converting unit 130 forproperly converting the signal derived from the bit expanding means intoan m-bit (for example, 10 bits) signal in response to the control signalderived from this control means 120.

[0068] The control signal output means 120 is constructed of an adder105 and a comparator 106. The converting unit 130 is constructed of alow-pass filter (LPF) 103, an LSB (least significant bit) extractingcircuit 104, adders 107 and 108, and a switch 108.

[0069] The output signal S2 of the 10-bit expanding circuit 102functioning as the bit expanding means is sent to the low-pass filter103 and the adder 107 of the converting unit 130, and to the adder 105of the control signal output means 120, respectively.

[0070] The low-pass filter 103 of the converting unit 130 executes afiltering process to the 10-bit processed image signal S2 to therebyoutput a signal S3. The output S3 of this low-pass filter 103 is sent tothe LSB extractor 104 and the adder 105 of the control signal outputmeans 120.

[0071] In the adder 105 of the control signal output means 120, adifference between the output signal S3 of the low-pass filter 103 andthe 10-bit processed output signal S2, namely S5=S2-S3 is outputted andthen is transferred to the comparator 106. The comparator 106 comparesthis difference with a predetermined threshold value, e.g., a value “4”corresponding to the 2 bits for addition. As will be discussed later,based upon a comparison result, the comparator 106 outputs a controlsignal C1 used to add a low order bit without losing the high frequencycomponent of the input image signal, and also another control signal C2used to control a way to add the low order bit.

[0072] The LSB extractor 104 of the control unit 130 extracts only 2bits from the 10-bits of the image signal on the LSB side as an outputsignal S4, and then supplied this output signal S4 to the switch 108.The control signal C1 derived from the comparator 106 is supplied as anON/OFF control signal. The control signal C2 is supplied to the adder107, and the output signal from the adder 107 is sent to the adder 109of the converting unit 130. The output signal from the switch 108 issupplied to the adder 109, and the output signal from the adder 109 isderived via an output terminal 110.

[0073] In FIG. 8, there is shown a concrete example of a circuitarrangement of the low-pass filter (LPF) 103 of the above-describedconverting unit 130.

[0074] In the filter arrangement shown in FIG. 8, an input signalsupplied to an input terminal 201 is sent to a 1 line delay circuit 202and an adder 204, and an output signal from the 1 line delay circuit 202is transferred to a 1 line delay circuit 203 and a 1 pixel delay element210. An output signal from the 1 line delay circuit is sent to an adder204 so as to be added with the above-explained input signal, and theadded signal is sent via a ½ coefficient multiplier 205 to a 1 pixeldelay element 206. An output signal from the 1 pixel delay element 206is supplied to an adder 207, and an output signal from the 1 pixel delayelement 210 is sent to the adder 207 and an output terminal 220. Anoutput signal from the adder 207 is supplied via a ½ coefficientmultiplier 208 to a 1 pixel delay element 209. An output signal from the1 pixel delay element 209 is sent to a 1 pixel delay element 211 and anadder 213. An output signal from the 1 pixel delay element 211 is sentvia a 1 pixel delay element 212 to an adder 213 and another adder 216.An output signal from the adder 213 is supplied via a ½ coefficientmultiplier 214 and a 1 pixel delay element 215 to an adder 216. Anoutput signal from the adder 216 is derived from an output terminal 219through a ½ coefficient multiplier 217 and a 1 pixel delay element 218.

[0075] The filter shown in FIG. 8 is a two-dimensional low-pass filter.FIG. 9A represents a 3×3 pixel block functioning as an input. Thetwo-dimensional low-pass filter extract a 3×3 pixel block around acertain pixel “e” to be processed. With respect to this object, thebelow-mentioned calculation outputs are recognized as the output valuesof the filter for the pixel “e”:a/16+b/8+c/16+d/8+e/4+f/8+g/16+h/8+i/16. In other words, 3×3, filtercoefficients for the respective pixels “a” to “i” are indicated in FIG.9B. From an output terminal 19, an output value processed in the filteris derived, and an original pixel value of the above-described inputsignal which is not filtered but has been delayed for a preselecteddelay time is derived from an output terminal 20.

[0076] It should be noted that although FIGS. 9A and 9B describe the FIRfilter with 3×3 taps, the number of taps may be arbitrarily selected.Alternatively, an IIR filter may be employed instead of this FRI filter.

[0077] Next, operation of the comparator 106 of the control signaloutput means 120 will now be explained. The comparator 106 firstcalculates an absolute value of the input signal S5. The comparator 106judges whether or not the absolute value of the input signal S5 islarger than a threshold value “4”. In this case, “4” of a 10-bit signalcorresponds to 1 bit of an 8-bit signal. Depending upon the judgementresult of the comparator 106, the comparator 106 outputs the controlsignal C1 to the switch 108 of the converting unit 130, and the controlsignal C2 to the adder 107.

[0078] The switch 108 is controlled in response to the control signalC1. When the absolute value of the input signal S5 is smaller than “4”,the switch 108 is turned ON to output the output signal S4 to the adder109. When the absolute value of the input signal S5 is greater than, orequal to “4”, the switch 108 is turned OFF. At this time, “0” isoutputted to the adder 109.

[0079] The above-described process will now be explained with referenceto FIG. 10A and FIG. 10B.

[0080]FIG. 10A represents a concrete example for the above-described8-bit input image signal S1 indicated by a solid line, and the outputsignal S3 from the low-pass filter 103, as indicated by a dot line. Inthis embodiment, 2 bits of the signal, on the LSB side thereof, producedfrom the low-pass filter 103 is added instead of 2 bits of the 10-bitsignal S2, on the LSD side thereof, obtained by expanding the inputsignal S1.

[0081] Since the signal S3 is produced by way of the low-pass filter,the high frequency component is lost to produce the blurred image. As aconsequence, 8 bits (on MSB side thereof) of the 10-bit image signal ofthe signal S2 is made coincident with the input image signal S1 in ordernot to lose the high frequency components as many as possible.

[0082] As a result, only when the difference between the respectivesignals S2 and S3 is smaller than a predetermined threshold value, forexample, “4”, 2 bits of the signal S3 on the LSB side is added insteadof 2 bits of the signal S2 on the LSB side. In other words, in such aflat region where a concentration change is small, the process operationis carried out such that 2 bits of the 10-bit image signal on the LSBside are replaced by 2 bits of the smoothed signal.

[0083] In the concrete example of FIG. 10A, a region A corresponds to aregion whose concentration change is small, in which since thedifference between the respective signals S2 and S3 is smaller than “4”,2 bits of the signal S3 on the LSB side is added to the signal S2. Tothe contrary, the region A1 corresponds to a region whose concentrationchange is large, and since the difference is larger than, or equal to 4,the signal S2 is directly outputted.

[0084] The adder 107 of FIG. 6 is controlled in response to the controlsignal C2. When the absolute value of the signal S5 is smaller than 4and the signal S5 is a positive integer (S2>S3), 4 is subtracted fromthe input signal S2 in the adder 107. In other cases, no processoperation is performed by the adder 107, but the input signal isoutputted therefrom. It is assumed that the output signal from thisadder 107 is “S6”.

[0085] The reason why the above-explained process operation is performedwill now be explained with reference to FIG. 10B. An “S2” indicated by asolid line shows such a signal S2 which is produced by bit-expanding the8-bit input signal S1 into the 10-bit input signal. “1(step)” in the8-bit signal S1 corresponds to “4(steps)” in the 10-bit expanded signalS2. An S3 indicated by a dot line of FIG. 10B is an output result of thelow-pass filter 3. This is because 2 bits (on the LSB side) of thesignal S3 produced by the low-pass filter 3 is added instead of 2 bitsof the signal S2 on the LSB side.

[0086] In the region A2, since S2<S3, 2 bits of the signal S3 on the LSBside is directly added to the signal S2. However, since the level of thesignal S2 is larger than the signal S3 in the region A1, namely S2>S3, 2bits of the signal S3 on the LSB side is added after “4” is previouslysubtracted from the signal S2.

[0087] Next, the output signal S6 obtained from the adder 107 issupplied to the adder 109. In this adder 109, the output signal S8 fromthe switch 108 is added to the output from the adder 107 to obtain anadded signal S7 (S7=S6+S8). This signal S7 is derived from the outputterminal 110.

[0088] As previously described, in the signal processing circuit of FIG.6, when the 8-bit quantized digital image signal is converted into the10-bit image signal, the smoothing process is performed at the sametime. Also, the high frequency component of the image signal when thesmoothing process is performed is not lost as many as possible.

[0089] That is, in the 10-bit image signal, an interval of 1 bit becomes{fraction (1/4 )} of an interval of 1 bit of the 8-bit image signal. Asa consequence, a more precise image can be represented, as compared withthe 8-bit image signal. Also, the 1-bit interval is not so easibleobservable.

[0090] Thus, as shown in FIG. 11A, when there is a 1-bit concentration(density) change in the 8-bit image signal, in the case that the 8-bitimage signal is bit-converted into the 10-bit image signal, thesmoothing process is performed thereto at the same time, so that such asmooth signal whose concentration change has been dispersed into 4pixels, as shown in FIG. 11B. Normally, when such a smoothing process iscarried out, the high frequency component of the image signal will belost. To avoid it, 8 bits (on the MSB side) of the smoothed 10-bit imagesignal are identical to 8 bits of the input image signal.

[0091] As a result, when the conventional method is carried out, namelythe 8-bit image signal is processed without any bit conversion, one sortof such a dither method for adding noise to the input image signal hasbeen employed. To the contrary, according to this embodiment, thequantizing noise, pseudo contour is relaxed without adding noise to theinput image signal.

[0092] In other words, according to the embodiment of the presentinvention, the 8-bit quantized digital image signal is converted intothe 10-bit image signal to produce the half tone (intermediategradation), so that the quantizing noise produced by the 8-bitquantizing process can be relaxed, or suppressed. In particular, thepseudo contour caused by the performance limit of the 8-bit image signalnever appear. In general, this may be realized by applying the presentembodiment to such a case that n bits are converted into m bits.

[0093] The image signal processing apparatus as shown in FIG. 6 may beapplied to, for example, a post filter 39 employed in a moving picturecoding/decoding apparatus shown in FIG. 15. The moving picturecoding/decoding apparatus shown in FIG. 15 employs the high-efficiencycoding technique for the moving picture with employment of linecorrelation and frame correlation. This high-efficiency coding techniquewill now be explained with reference to FIG. 12 to FIG. 20.

[0094] When the line correlation is utilized, an image signal may becompressed by performing, for example, the DCT (discrete cosinetransform) process.

[0095] When the frame correlation is utilized, the image signal may befurther compressed to be coded.

[0096] For instance, as shown in FIG. 12A, when frame images PC1, PC2,PC3 are produced at time instants t=t1, t2, t3, respectively, adifference between image signals of the frame images PC1 and PC2 iscalculated to thereby form an image PC12 as indicated in FIG. 12B. Also,another difference between the frame images PC2 and PC3 of FIG. 12A iscalculated to produce an image PC23 of FIG. 12B. Normally, sincetemporally adjoining frame images do not contain a large change, when adifference between the temporally successive frame images is calculated,a difference signal is a small value. That is, as to the image PC12shown in FIG. 12B, a signal of such a portion indicated by a hatchedline in the image PC12 of FIG. 12B is obtained as a difference betweenthe frame image signals of the frame images PC2 and PC3 in FIG. 12A.Then, when this difference signal is coded, the coding amount can becompressed.

[0097] However, if only the above-described difference signal istransmitted, then the original image cannot be recovered. Therefore,while the images of the respective frames are set to any one of an Ipicture (Intra-coded picture), a P picture (Predictive-coded picture),and a B picture (Bidirectionally-predictive-coded picture), the imagesignal is compressed/coded.

[0098] In other words, as represented in FIG. 13A and FIG. 13B, imagesignals of 17 frames defined from a frame F1 to a frame F17 are handleas a group of picture, namely, one unit of the coding process. Then, theimage signal of the head frame F1 is coded as the I picture, the imagesignal of the second frame F2 is coded as the B picture, and the imagesignal of the third frame F3 is coded as the P picture. Subsequently,the image signals from the fourth frame F4 until the last frame F17 arealternately processed as either the B pictures or the P pictures.

[0099] As the image signal of the I picture, the image signal for 1frame thereof is directly transmitted without any process. To thecontrary, as the image signal of the P picture, basically, as indicatedin FIG. 13A, a difference between the image signals of either the Ipictures, or the P pictures, which temporally precede this I pictureimage signal is coded to be transferred. Furthermore, as the imagesignal of the B picture, basically, as indicated in FIG. 13B, adifference is calculated from average values of the frame images whichtemporally precede, or succeed this I picture image signal, and thenthis calculated difference is coded to be transmitted.

[0100]FIG. 14A and FIG. 14B schematically represent the principle ideaof the method for coding a moving picture (image) signal. In FIG. 14A,frame data of the moving picture signal is schematically shown, whereasin FIG. 14B, frame data to be transmitted is schematically shown. Asrepresented in FIG. 14A and FIG. 14B, since the first frame F1 isprocessed as the I picture, namely the non-interpolated frame, thisfirst frame F1 is transmitted as transmit data FIX (transmitnon-interpolated frame data) (namely, intra-coding process). To thecontrary, since the second frame F2 is processed as the B picture, i.e.,an interpolated frame, a calculation is carried out about a differencecomponent between the temporally preceding frame F1 and the averagevalue of the temporally succeeding frame F3 (frame codednon-interpolated frame). This difference is transmitted as transmit data(transmit interpolated frame data).

[0101] Precise speaking, as this B picture process, there are four sortsof modes switchable in unit of a macroblock. As the first process, thedata about the original frame F2 is directly transmitted as transmitdata F2X as indicated by an arrow SP1 of a broken line in FIG. 14(intra-coded mode), which is similar to the process for the I picture.As the second process, a difference between the second frame F2 and thetemporally succeeding frame F3 is calculated, and then this differenceis transmitted as indicated by an arrow SP2 of a broken line in thisdrawing (backward prediction mode). As the third process, the differencebetween the second frame F2 and the temporally preceding frame F1 istransmitted as indicated by an arrow SP3 of a broken line in thisdrawing (forward prediction mode). Furthermore, as the fourth process,the difference value between the temporally preceding frame F1 and theaverage value of the temporally succeeding frame F3 is produced, andthis difference value is transmitted as transmit data F2X (bidirectionalprediction mode).

[0102] The method by which the most least data is transmitted amongthese fourth methods is employed in unit of a macroblock.

[0103] It should be noted that when the difference data is transmitted,either a movement vector X1 between the frame images (predicted images)used to calculate the difference value (namely, a movement vectorbetween the frames F1 and F2 in case of the forward prediction mode), oranother movement vector X2 (a movement vector between the frames F3 andF2 in case of the backward prediction mode), otherwise both of themovement vectors X1 and X2 (in case of the bidirectional predictionmode) are transmitted together with the difference data.

[0104] With respect to the frame F3 of the I picture (frame-codednon-interpolated frame), while using the temporally preceding frame F1as the prediction image, the difference signal (indicated by an arrowSP3 of a broken line) between this frame F1 and the frame F3 iscalculated, and also the movement vector X3 is calculated. This istransmitted as the transmit data F3X (forward prediction mode).Alternatively, the data on the original frame F3 is directly transmittedas the transmit data F3X (indicated as an arrow SP1 of a broken line)(intra-coding mode). In this P picture, the data transmit method isselected in a similar manner to the B picture, namely, the method fortransmitting the most least data is selected in unit of a macroblock.

[0105] It should be noted that both of the frame F4 in the B picture andthe frame F5 in the P picture are processed in a similar manner to theabove-described case to thereby obtain transmit data F4X, F5X, andmovement vectors X4, X5, X6 etc.

[0106] Next, FIG. 15 schematically represents a structural example of anapparatus for coding a moving picture signal to transmit the codedmoving picture signal, and also for decoding this coded moving picturesignal based upon the above-described principle idea.

[0107] In FIG. 15, a coding apparatus 1 codes an inputted video(picture) signal and transmits the coded video signal to a recordingmedium 3 as a transmission path in order to be recorded thereon. Then, adecoding apparatus 2 reproduces the signal recorded on the recordingmedium 3 and decodes this reproduced signal to be outputted.

[0108] First, in the coding apparatus 1, the video signal VD entered viathe input terminal 10 is inputted into a preprocessing circuit 11 bywhich a luminance signal and a color signal (color difference signal inthis case) are separated. The separated luminance signal and colorsignal are A/D-converted by A/D converters 12 and 13, respectively. Thedigital video signals A/D-converted in the A/D converters 12 and 13 aresupplied to either a front-end filter, or a prefilter 19 so as to befiltered. Thereafter, the filtered digital video signals are supplied toa frame memory 14 so as to be stored therein. In this frame memory 14,the luminance signal is stored in a luminance frame memory 15, and thecolor difference signal is stored in a color difference signal framememory 16, respectively.

[0109] The above-explained front-end filter, or prefilter 19 performssuch a process operation to improve the coding efficiency and the imagequality. This prefilter 19 correspond to, for example, a noiseeliminating filter, or a filter for limiting a bandwidth. As a concreteexample of this prefilter, the two-dimensional low-pass filter with the3×3 pixels may be employed, as explained in connection with FIG. 8 andFIG. 9. In this filter, the uniform filtering process is continuouslyperformed irrelevant to the input image signal, the conditions of thedecoders.

[0110] A format converting circuit 17 converts a frame format signalstored in a frame memory 14 into a block format signal. That is, asshown in FIG. 16, the video signal stored in the frame memory 14 isdetermined as such frame format data 111 that V lines are collected and1 line contains H dots. The format converting circuit 17 subdivides 1frame signal into N pieces of slices 112, while using 16 lines as aunit. Then, each slice 112 is subdivided into M pieces of macroblocks.Each macroblock 113 is constructed of a luminance signal correspondingto 16×16 pixels (dots), and this luminance signal is further subdividedinto blocks Y[1] to Y[4], and each block is constituted by 8×8-dot Cbsignal and a 8×8-dot Cr signal.

[0111] As described above, the data converted into the block format issupplied from the format converting circuit 17 into the encoder 18 so asto be encoded. A detailed encoding operation will be discussed laterwith reference to FIG. 17.

[0112] The signal encoded by the encoder 18 is outputted to thetransmission path as a bit stream which will then be recorded on, forexample, the recording medium 3.

[0113] The data reproduced from the recording medium 3 is supplied tothe decoder 31 of the decoding apparatus 2 so as to be decoded. Adetailed structure of the decoder 31 will be explained with reference toFIG. 20.

[0114] The data decoded by the decoder 31 is entered into the formatconverting circuit 32, so that the block format is transformed into theframe format. Then, the luminance signal in the frame format is suppliedto a luminance signal frame memory 33 so as to be stored therein, andthe color difference signal is supplied to a color difference signalframe memory 35 in order to be stored therein. Both of the luminancesignal and the color difference signal, which are read out from theluminance signal frame memory 34 and the color difference signal framememory 35 are furnished to either a post-staged filter, or a post filter39 so as to be filtered. Thereafter, the filtered signals areD/A-converted by D/A converters 36 and 37 into analog signals. Theanalog signals are supplied to a postprocessing circuit 38 in order tobe combined with each other. The output picture (image) signal issupplied from an output terminal 30 to a display such as a CRT (notshown) for display operation.

[0115] This post filter 39 performs a process to improve an imagequality, namely is employed so as to mitigate image deterioration causedby coding the image. In general, as this post filter 39, such a filteris utilized which removes, for instance, block distortion, noiseproduced near a sharp edge, or the quantizing noise. As explained withreference to FIG. 8 and FIG. 9, the two-dimensional low-pass filter with3×3 pixels is utilized. In this embodiment, the image signal processingapparatus shown in FIG. 6 is employed.

[0116] In other words, the luminance signal read from the luminancesignal frame memory 34, and the color difference signal read from thecolor difference signal frame memory 35 are entered into the inputterminal 101 of FIG. 6. The apparatus of FIG. 6 processes at least oneof these luminance signal and color difference signal, or both of thesesignals in a parallel manner, or a time-divisional manner. Then, thisapparatus derives the processed signal from the output terminal 110 andsupplies the derived signal to the D/A converters 36 and 37.

[0117] Referring now to FIG. 17, an arrangement of the encoder 18 willbe described.

[0118] The image data which has been supplied via the input terminal 49and should be encoded is entered into a movement vector detectingcircuit 50 in unit of macroblock. The movement vector detecting circuit50 processes the image data of the respective frames as an I picture, aP picture, or a B picture in accordance with a preset sequence. Adecision how to process the image data of each frame as the I picture, Ppicture, or B picture is previously made (for example, a shown in FIG.13, a group of picture constituted by the frame F1 to the frame F17 isprocessed as I, B, P, B, P, - - - , B, P).

[0119] The image data of such a frame to be processed as the I picture(e.g., frame F1) is transferred from the movement vector detectingcircuit 50 to a forward original image unit 51 a of the frame memory 51so as to be stored therein. The image data of such a frame to beprocessed as the B picture (e.g., frame F2) is transferred to anoriginal image unit (reference original image unit) 51 b so as to bestored therein. The image data to be processed as the P picture (forexample, frame F3) is transferred to a backward original image unit 51 cin order to be stored therein.

[0120] At the next timing, when the image data of such a frame to beprocessed as either the B picture (e.g., frame F4), or the P picture(e.g., frame F5) is entered, the image data of the first P picture(namely, frame F3) which has been stored in the backward original imageunit 51 c is transferred to the forward original image unit 51 a. Then,the image data of the next B picture (frame F4) is stored in theoriginal image unit 51 b (namely, overwritten), and the image data ofthe next P picture (frame F5) is stored in the backward original imageunit 51 c (namely, overwritten). Such an operation is sequentiallyrepeated.

[0121] The signals of the respective picture stored in the frame memory51 are read out from this frame memory 51, and the read signals areprocessed in a prediction mode switching circuit 52 by way of either aframe prediction mode process, or a field prediction mode process.Moreover, under control of a prediction judging circuit 54, thecalculations are performed in a calculation unit 53 in accordance withthe intra coding mode, the forward prediction mode, the backwardprediction mode, or the bidirectional prediction mode. A decision as towhich process operation is carried out is made in unit of macroblock inresponse to a prediction error signal (namely, a difference between areference image to be processed and a predicted image thereof). As aconsequence, the movement vector detecting circuit 50 produces anabsolute value summation (otherwise, squared summation) of theprediction error signals employed in this judgement, and an evaluatedvalue of the intra coding mode corresponding to the prediction errorsignal in unit of macroblock.

[0122] Now, the frame prediction mode and the field prediction mode inthe prediction mode switching circuit 52 will be explained.

[0123] When the frame prediction mode is set, the prediction modeswitching circuit 52 directly outputs four luminance blocks Y[1] to Y[4]supplied from the movement vector detecting circuit 50 to a post-stagedcalculation unit 53. That is, in this case, as represented in FIG. 18,the data of the lines in the odd-numbered field, and the data of thelines in the even-numbered field are mixed with each other in therespective luminance blocks. It should be noted that a solid line ineach of macroblocks of FIG. 18A and FIG. 18B indicates the data of thelines in the odd-numbered field (line of a first field), and a brokenline thereof shows the data of the lines in the even-numbered field(line of a second field). Symbols “a” and “b” of FIG. 18A and FIG. 18Bshow units of movement compensation. In this frame prediction mode,prediction is carried out by using the four luminance blocks(macroblocks) as one unit, and one movement vector corresponds to thefour luminance blocks.

[0124] To the contrary, when the field prediction mode is set, theprediction mode switching circuit 52 outputs such a signal inputted fromthe movement vector detecting circuit 50 having the structure shown inFIG. 18A to the calculation unit 53 in such a manner that, as indicatedin FIG. 18B, the luminance blocks Y[1] and Y[2] among the four luminanceblocks are arranged only by the data of the lines in the odd-numberedfield, whereas the remaining two luminance blocks Y[3] and Y[4] arearranged by the data of the lines in the even-numbered field. In thiscase, one movement vector corresponds to the two luminance blocks Y[1]and Y[2], whereas another movement vector corresponds to the remainingtwo luminance blocks Y[3] and Y[4].

[0125] The color difference signal is supplied to the calculation unit53 in the case of the frame prediction mode, as shown in FIG. 18, undersuch a condition that the data of the lines in the odd-numbered fieldand the data of the lines in the even-numbered field are mixed with eachother. In the case of the field prediction mode, as represented in FIG.18B, the upper half portions (4 lines) of the respective colordifference blocks Cb and Cr and the color difference signals in theodd-numbered field corresponding to the luminance blocks Y[1] and Y[2],and the lower half portions thereof (4 lines) are the color differencesignals in the even-numbered fields corresponding to the luminanceblocks Y[3] and Y[4].

[0126] The movement vector detecting circuit 50 produces the evaluationvalue in the intra coding mode, and the absolute value summation of therespective prediction errors in unit of macroblock. These values areused in the prediction judging circuit 54 to determine that any one ofthe intra coding mode, the forward prediction mode, the backwardprediction mode, and the bidirectional prediction mode is employed toperform the prediction operation with respect to the respectivemacroblock, and also to determine that either the frame prediction mode,or the field prediction is used to execute the process operation.

[0127] In other words, as the evaluation value of the intra coding mode,a calculation is made of an absolute value summation “Σ|Aij-(averagevalue of Aif)|” of differences between the signal Aij of the macroblockof the reference image which will be coded, and an average valuethereof. As an absolute value summation of the forward predictionerrors, another calculation is made of a summation Σ|Aij-Bij| of anabsolute value |Aij-Bij| about a difference (Aij-Bij) between the signalAij of the macroblock of the reference image and the signal Bij of themacroblock of the prediction image. Similar to the above-explainedforward prediction case, as an absolute summation of the predictionerrors for the backward prediction and the bidirectional prediction,these absolute value summations are calculated with regard to the frameprediction mode and the field prediction mode (the relevant predictionimage is changed into the different prediction image from that of theforward prediction).

[0128] These absolute value summations are supplied to the predictionjudging circuit 54. The prediction judging circuit 54 selects thesmallest ne from the absolute value summations of the prediction errorsof the forward prediction, backward prediction, and bidirectionalprediction in each of the frame prediction mode and the field predictionmode, as an absolute value summation of prediction errors in the interprediction. Furthermore, the prediction judging circuit 54 compares thisabsolute value summation of the prediction errors in this interprediction with the evaluation value of the intra coding mode to therebyselects a smaller absolute value summation. Then, the mode correspondingto this selected value is selected as the prediction mode and theframe/field prediction mode. That is, when the evaluation value of theintra coding mode becomes smaller, the intra coding mode is set. Whenthe absolute value summation of the prediction errors of the interprediction becomes smaller, such a mode whose corresponding absolutevalue summation is the smallest among the forward prediction, thebackward prediction, and the bidirectional prediction is set as theprediction mode, and the frame/field prediction mode.

[0129] As previously explained, the prediction mode switching circuit 52supplies the signal of the macroblock of the reference image to thecalculation unit 53 with such an arrangement as shown in FIG. 18corresponding to the mode selected by the prediction judging circuit 54among the frame or field prediction mode. The movement vector detectingcircuit 50 outputs the movement vector between the prediction image andthe reference image, which corresponds to the prediction mode selectedby the prediction judging circuit 54, and supplies the movement vectorto a variable length coding circuit 58 and a movement compensatingcircuit 64 (will be discussed later). As this movement vector, aselection is made of such a movement vector in which the absolute valuesummation of the corresponding prediction error becomes minimum.

[0130] The prediction judging circuit 54 sets the intra coding mode(namely, mode with no movement compensation) is set as the predictionmode when the image data of the I picture is read out from the forwardoriginal image unit 51 a by the movement vector detecting circuit 50,and changes the switch 53 d of the calculation unit 53 to the contact“a”. As a result, the image data of the I picture is inputted to a DCTmode switching circuit 55.

[0131] This DCT mode switching circuit 55 outputs the data of the fourluminance blocks to the DCT circuit 56, as shown in FIG. 19A, or FIG.14B, under either a condition that the lines of the odd-numbered fieldand the lines of the even-numbered field are mixed with each other, or aseparated condition (field DCT mode).

[0132] In other words, the DCT mode switching circuit 55 compares thecoding efficiency achieved when the data is DCT-processed by mixing thedata of the odd-numbered fields with the data of the even-numberedfields with the coding efficiency achieved when the data isDCT-processed under separate condition, thereby selecting the mode withthe better coding efficiency.

[0133] For example, as indicated in FIG. 19A, while the inputted signalis so arranged that the lines of the odd-numbered field and the lines ofthe even-numbered field are-mixed, a calculation is done to obtain adifference between the signals of the lines in the add-numbered fieldand the signals of the lines in the even-numbered field located adjacentto those of the odd-numbered field along the vertical direction.Furthermore, a summation of the absolute values (or squared summation)is calculated. Also, as indicated in FIG. 19B, while the inputted signalis so arranged that the lines of the odd-numbered field and the lines ofthe even-numbered field are separated, a calculation is done to obtain adifference between the signals of the mutual lines in theodd-numbered-field and another difference between the signals of themutual lines in the even-numbered field located adjacent to each otheralong-the vertical direction. Furthermore, a summation of the absolutevalues (or squared summation) is calculated. In addition, both of theseabsolute value summations are compared with each other, and the DCT modecorresponding to the smaller value is set. In other words, when theformer value becomes smaller, the frame DCT mode is set, whereas whenthe latter value becomes smaller, the field DCT mode is set.

[0134] Then, the data having the structure corresponding to the selectedDCT mode is outputted to the DCT circuit 56, and further the DCT flagindicative of the selected DCT mode is outputted to the variable lengthcoding circuit 58.

[0135] As apparent from the comparison between the frame/fieldprediction mode (see FIG. 18A and FIG. 18B) in the prediction modeswitching circuit 52 and the DCT mode (see FIG. 19A and FIG. 19B) inthis DCT mode switching circuit 55, the data structures in both of themodes re substantially identical to each other with respect to theluminance blocks.

[0136] When the frame prediction mode (namely, mode in whichodd-numbered lines and even-numbered lines are mixed) is selected in theprediction mode switching circuit 52, there are higher possibilitiesthat the frame DCT mode (namely, mode in which odd-numbered lines andeven-numbered lines are mixed) is selected also in the DCT modeswitching circuit 55. Similarly, when the field prediction mode (namely,mode in which data in odd-numbered lines are separated from data ineven-numbered lines) is selected in the prediction mode switchingcircuit 52, there are higher possibilities that the field DCT mode(namely, mode in which data in odd-numbered field are separated fromdata in even-numbered field) is selected in the DCT mode switchingcircuit 55.

[0137] However, the above-described conditions are not alwaysestablished, but the mode is selected in such a manner that the absolutevalue summation of the prediction errors becomes small in the predictionmode switching circuit 52, whereas the mode is determined in such a waythat the coding efficiency becomes high in the DCT mode switchingcircuit 55.

[0138] The image data on the I picture outputted from the DCT modeswitching circuit 55 is inputted into the DCT circuit 56 to beDCT-processed (discrete cosine transform), thereby being transformedinto the DCT coefficient. This DCT coefficient is inputted into thequantizing circuit 57, so as to be quantized by the quantizing stepcorresponding to the data storage amount of the transmit buffer 59(buffer storage amount). Thereafter, the quantized DCT coefficient isentered into the variable length coding circuit 58.

[0139] The variable length coding circuit 58 converts the image data(data on I picture in this case) supplied from the quantizing circuit 57in correspondence with the quantizing step (scale) supplied from thequantizing circuit 57 into such a variable length code as the Huffmancode, and then outputs the variable length code to the transmit buffer59.

[0140] Also, to the variable length coding circuit 58, the quantizingstep (scale) is supplied from the quantizing circuit 57; the predictionmode (namely, mode indicating that any one of intra coding mode, forwardprediction mode, backward prediction mode, and bidirectional predictionmode is set) is inputted from the prediction judging circuit 54; themovement vector is entered from the movement vector detecting circuit50; and the prediction flag (namely, flag indicating that any one offrame prediction mode and field prediction mode) is inputted from theprediction judging circuit 54; and also the DCT flag (namely, flagindicating that any one of flame DCT mode and field DCT mode is set)outputted from the DCT mode switching circuit 55 is entered. Theseparameters are similarly converted into variable length codes.

[0141] The transmit buffer 59 temporarily stores the inputted data, andsupplies the data corresponding to the data stored amount thereof to thequantizing circuit 57.

[0142] In the case that the data remaining amount of the transmit buffer59 is increased up to the allowable upper limit value, the transmitbuffer 59 lowers the data amount of the quantizing data by increasingthe quantizing scale of the quantizing circuit 57 in response to thequantizing control signal. Conversely, when the data remaining amount isreduced up to he allowable lower limit value, the transmit buffer 59increases the data amount of the quantizing data by decreasing thequantizing scale of the quantizing circuit 58 in response to thequantizing control signal. Thus, overflows or underflows of the transmitbuffer 59 can be avoided.

[0143] Then, the data stored in the transmit buffer 59 is read out at apreselected timing, and then outputted via the output terminal 69 to thetransmission path, thereby being recorded on, for example, the recordingmedium 3.

[0144] On the other hand, the data of the I picture outputted from thequantizing circuit 57 is inputted into a dequantizing circuit 60 so asto be dequantized in accordance with the quantizing step supplied fromthe quantizing circuit 57. The output from the dequantizing circuit 60is inputted into an IDCT (inverse DCT) circuit 61 in order to beinverse-DCT-processed. Thereafter, the resulting signal is supplied viaa calculator 62 to the forward prediction image unit 63 a of the framememory 63 so as to be stored.

[0145] When the image data of the respective frames sequentially enteredinto the movement vector detecting circuit 50 are processed as thepictures of I, B, P, B, P, B - - - (as explained before) by thismovement vector detecting circuit 50, the image data of the firstlyinputted frame is processed as the I picture. Thereafter, the image dataof the thirdly inputted frame is processed as the P picture before theimage data of the secondly entered frame is processed as the I picture.The reason is such that as the B picture owns a certain possibility withthe backward prediction and the bidirectional prediction, if the Ppicture as the backward prediction image is not prepared in advance,then this Z picture cannot be decoded.

[0146] Under such a circumstance, the movement vector detecting circuit50 commences the process operation of the image data on the P picturestored in the backward original image unit 51 c subsequent to theprocess operation of the I picture. Then, similar to the above-explainedcase, the absolute value summation of the frame differences (predictionerror differences), and the evaluate value of the intra coding mode inunit of macroblock are supplied from the movement vector detectingcircuit 50 to the prediction judging circuit 54. In response to theevaluation value of the intra coding mode for this P picture in unit ofmacroblock and also the absolute value of the prediction errors, theprediction judging circuit 54 sets any one of the frame prediction mode,and any one of the intra coding mode and the forward prediction mode.

[0147] When the intra coding mode is set, the calculation unit 53changes the switch 53 d to the contact “a”, as explained above. As aresult, similar to the data on the I picture, this data is transferredto the transmission path via the DCT mode switching circuit 55, the DCTcircuit 56, the quantizing circuit 57, the variable length codingcircuit 58, and the transmit buffer 59. This data is furnished via thedequantizing circuit 60, the EDCT circuit 61, and the calculator 62 tothe backward prediction image unit 63 b of the frame memory 63 so as tobe stored therein.

[0148] On the other hand, when the forward prediction mode is selected,the switch 53 b is changed into the contact “b”, and also the image data(image data of I picture in this case) stored in the forward predictionimage unit 63 a of the frame memory 63 is read out. Then, the movementis compensated by the movement compensating circuit 64 in response tothe movement vector outputted from the movement vector detecting circuit50. In other words, when the prediction judging circuit 54 instructs toset the forward prediction mode, the movement compensating circuit 64shifts the read address of the forward-prediction image unit 63 a onlyby such a shift value corresponding to the movement vector from aposition corresponding to the position of the macroblock presentlyoutputted by the movement detecting circuit 50, thereby reading out thedata to produce the prediction image data.

[0149] The prediction image data outputted from the movementcompensating circuit 64 is supplied to the calculator 53 a. Thecalculator 53 a subtracts the prediction image data corresponding to themacroblock of the reference image data, which is supplied from themovement compensating circuit 64, from the data on this reference imagein the macroblock supplied from the prediction mode switching circuit52. Then, this calculator 53 a outputs a difference thereof (predictionerror). This difference data is transmitted via the DCT mode switchingcircuit 55, the DCT circuit 56, the quantizing circuit 57, the variablelength coding circuit 58, and the transmit buffer 59 to the transmissionpath. This difference data is locally decoded by the dequantizingcircuit 60 and the IDCT circuit 61 to thereby be entered into thecalculator 62. The data identical to the prediction image data suppliedto the calculator 53 a is furnished to this calculator 62. Thecalculator 62 adds the prediction image data outputted from the movementcompensating circuit 64 to the difference data outputted by the IDCTcircuit 61. As a result, the image data of the original (decoded) Ppicture is obtained. This image data of the P picture is supplied to thebackward prediction image unit 63 b of the frame memory 63, so as to bestored therein. It should be understood that since the data structure ofthe difference data outputted by the IDCT circuit must be actuallyidentical to the data structure of the prediction image data, which aresupplied to the calculator 62, such a circuit is required whichrearranges the data so as to accept such a case that the frame/fieldprediction mode and the frame/field DCT modes are different. However,for the sake of simplicity, this circuit is omitted.

[0150] As explained above, the movement vector detecting circuit 50executes-the process operation of the B picture after the image data ofthe I picture and the P picture have been stored into the forwardprediction image unit 63 a and the backward prediction image unit 63 b,respectively. In correspondence with the evaluation value of the intracoding mode in unit of macroblock, and also the dimension of theabsolute value summation between the frame differences, the predictionjudging circuit 54 sets the frame/field prediction mode, and also setsthe prediction mode to any one of the intra coding mode, the forwardprediction mode, the backward prediction mode, and the bidirectionalprediction mode.

[0151] As previously described, when either the intra coding mode, orthe forward prediction mode is selected, the switch 53 d is switched toeither the contact “a” or the contact “b”. At this time, a similarprocess operation is carried out as in the P picture, and the data istransmitted.

[0152] To the contrary, when either the backward prediction mode or thebidirectional prediction mode is set, the switch 53 d is changed intothe contact “c” of the contact “d”.

[0153] In the backward prediction mode where the switch 53 d is switchedto the contact “c”, the image data (image data on I picture in thiscase) stored in the backward prediction image unit 63 b of the framememory 63 is read out. Then, the movement is compensated by the movementcompensating circuit 64 in response to the movement vector outputtedfrom the movement vector detecting circuit 50. In other words, when theprediction judging circuit 54 instructs to set the backward predictionmode, the movement compensating circuit 64 shifts the read address ofthe backward prediction image unit 63 b only by such a shift valuecorresponding to the movement vector from a position corresponding tothe position of the macroblock presently outputted by the movementdetecting circuit 50, thereby reading out the data to produce theprediction image data.

[0154] The prediction image data outputted from the movementcompensating circuit 64 is supplied to the calculator 53 b. Thecalculator 53 b subtracts the prediction image data which is suppliedfrom the movement compensating circuit 64, from the data on thisreference image in the macroblock supplied from the prediction modeswitching circuit 52. Then, this calculator 53 b outputs a differencethereof (prediction error). This difference data is transmitted via theDCT mode switching circuit 55, the DCT circuit 56, the quantizingcircuit 57, the variable length coding circuit 58, and the transmitbuffer 59 to the transmission path.

[0155] In the bidirectional prediction mode where the switch 53 d isswitched to the contact “d”, both of the image data (I picture imagedata in this case) stored into the forward prediction image unit 63 aand the image data (P picture image data in this case) stored into thebackward prediction image unit 63 b are read out. The movements of theseread data are compensated by the movement compensating circuit 64 naccordance with the movement vector outputted from the movement vectordetecting circuit 50. In other words, when the prediction judgingcircuit 54 instructs to set the bidirectional prediction mode, themovement compensating circuit 64 shifts the read addresses of theforward prediction image unit 63 a and the backward prediction imageunit 63 b by such a value corresponding to the movement vector from theposition corresponding to the position of the macroblock presentlyoutputted by the movement vector detecting circuit 50, thereby readingout the data to produce the prediction image data. This movement vectorbecomes two vectors for the forward prediction image and the backwardprediction image in the frame prediction mode, and becomes 4 vectors,namely two vectors for the forward prediction image, and two vectors forthe backward prediction image in the field prediction mode.

[0156] The prediction image data outputted from the movementcompensating circuit 64 is supplied to the calculator 53 c. Thecalculator 53 c subtracts the average value of the prediction image datasupplied from the movement compensating circuit 64 from the data of thereference image supplied from the movement vector detecting circuit 50in the macroblock, and outputs this difference. The difference data istransmitted via the DCT mode switching circuit 55, the DCT circuit 56,the quantizing circuit 57, the variable length coding circuit 58, andthe transmit buffer 59 to the transmission path.

[0157] Since the B picture image is not used as the prediction image forother image, this B picture image is not stored in the frame memory 6.

[0158] In the frame memory 63, the forward prediction image unit 63 aand the backward prediction image unit 63 b are switched, if necessary.The image data stored in either one image unit 63 a, or the other imageunit 63 b with respect to a preselected reference image, may be switchedas the forward prediction image, or the backward prediction image to beoutputted.

[0159] Although the previous description has been made about theluminance blocks, this processing operation is similarly applied to thecolor difference blocks in unit of macroblocks shown in FIG. 18 and FIG.19, and then the processed data is transmitted. It should be noted thatas the movement vector used to process the color difference block, themovement vector of the luminance block corresponding thereto issubdivided into ½ along the vertical direction and the horizontaldirection.

[0160] Next, FIG. 20 schematically shows a block diagram of anarrangement of the decoder 31 indicated in FIG. 15. The decoded imagedata transmitted via the transmission path (recording medium 3) isreceived by a receiver circuit (not shown), is reproduced in thereproducing apparatus, and is temporarily stored into a receiver buffer81 via an input terminal 80. Thereafter, this image data is supplied toa variable length decoding circuit 82 of a decoder circuit 90. Thevariable length decoding circuit 82 variable-length-decodes the datasupplied from the receiver buffer 81, outputs a movement vector, aprediction mode, a prediction flag, and a DCT flag to a movementcompensating circuit 87. Also, the variable length decoding circuit 82outputs a quantizing step to a dequantizing circuit 83, and also thedecoded image data to the dequantizing circuit 83.

[0161] The dequantizing circuit 83 dequantizes the image data suppliedfrom the variable length decoding circuit 82 in accordance with thequantizing step supplied from the variable length decoding circuit 82,to thereby supply the resultant data to an IDCT circuit 84. The data(DCT coefficient) outputted from the dequantizing circuit 83 isinverse-DCT processed in the IDCT circuit 84 to be supplied to acalculator 85.

[0162] In the case that the image data supplied from the IDCT circuit 84is the data of the I picture, this data is outputted from the calculator85, and also is supplied to a forward prediction image unit 86 a of aframe memory 86 to be stored therein in order to produce predictionimage data of such image data (image data of P or B picture) which willbe entered into the calculator 85 later. This data is outputted to theformat converting circuit 32 (see FIG. 15).

[0163] When the image data supplied from the EDCT circuit 84 correspondsto the data on the I picture where the preceding image data by 1 frameis the prediction image data; and also to the data in the macroblockencoded in the forward prediction mode, the preceding image data by 1frame (data on I picture) which has been stored in the forwardprediction image unit 86 a of the frame memory 86 is read out. This readdata is compensated by the movement compensating circuit 87 incorrespondence with the movement vector outputted from the variablelength decoding circuit 82. Then, in the calculator 85, this compensateddata is added to the image data (data of difference) supplied from theIDCT circuit 84. The added data derived from the calculator 85, namelythe decoded data of the P picture is stored into a backward predictionimage unit 86 b of the frame memory 86 in order to produce predictionimage data of such image data (image data of B picture, or P picture)which will be inputted into this calculator 85 later.

[0164] Even when the data of the P picture is inputted, the data of themacroblock coded in the intra coding mode is not especially processed inthe calculator 85, but is directly stored into the backward predictionimage unit 86 b in a similar manner to the data of the I picture.

[0165] Since this P picture corresponds to such an image which should bedisplayed subsequent to the next B picture, this P picture is not yetoutputted to the format converting circuit 32 at this time (aspreviously explained, P picture inputted after B picture is processedprior to B picture to be transmitted).

[0166] When the image data supplied from the IDCT circuit 84 is the dataof the I picture, in response to the prediction mode supplied from thevariable length decoding circuit 82, the image data of the I picture (incase of forward reduction mode) stored in the forward prediction imageunit 86 a of the frame memory 86 is read out. The image data of the Ppicture (in case of backward prediction mode) stored in the backwardprediction image unit 86 b is read out. Otherwise, both of these imagedata (in case of bidirectional prediction mode) are read out. Then, themovement corresponding to the movement vector outputted from thevariable length decoding circuit 82 is compensated in the movementcompensating circuit 87, so that a prediction image is produced. Itshould be understood that when no movement compensation is required (incase of intra coding mode), such a prediction image is not produced.

[0167] As a described above, the data movement-compensated by themovement compensating circuit 87 is added with the output from thecalculator 85. The addition result is outputted via an output terminal91 to the format converting circuit 32.

[0168] It should also be noted that this addition result corresponds tothe data of the B picture, and since this image data is not utilized soas to produce a prediction image of other image, this image data is notstored into the frame memory 86.

[0169] After the image of the B picture has been outputted, the imagedata of the P picture stored in the backward prediction image unit 86 bis read out and then is outputted as the reproduction image via themovement compensating circuit 87 and the calculator 85. At this time,neither the movement compensation, nor the addition is carried out.

[0170] It should be noted that there are not shown such circuitscorresponding to the prediction mode switching circuit 52 and the DCTmode switching circuit 55 employed in the encoder 18 of FIG. 17 in thisdecoder 13, the process operations executed by these circuits areperformed by the movement compensating circuit 87, namely the structurethat the signals of the lines in the odd-numbered field are separatedfrom the signals of the lines in the even-numbered field is returned tothe original mixing structure, if necessary.

[0171] Although the luminance signals have been processed in theabove-described embodiment, the color difference signals may besimilarly processed. In this case, as the movement vector the movementvector for the luminance signal is subdivided into ½ along the verticaldirection and the horizontal direction.

[0172] Referring now to FIG. 21, another embodiment of the presentinvention will be explained.

[0173] It should be noted that a moving picture coding/decodingapparatus of this embodiment shown in FIG. 21 is similar to theabove-described apparatus represented in FIG. 15, and thereafter may beapparently applied as the post filter 39 of FIG. 15.

[0174] In FIG. 21, when an 8-bit image signal S11 is inputted to aninput terminal 301, this image signal is supplied via a noise removingfilter 300 (will be discussed later, if necessary) to a 10-bit expandingcircuit 302, 2 bits of “0” are added to the LSB of the input 8-bit imagesignal so as to expand the bit of this image signal, thereby producing a10-bit output signal S12. This 10-bit image signal S12 is supplied to acontrol signal output means 320 and a converting unit 330. These signalsS11 and S12 are similar to the signals S1 and S2 of the above-describedembodiment shown in FIG. 6.

[0175] The output signal S12 from the 10-bit expanding circuit 302 isinputted into a low-pass filter 303 and a multiplier 306 employed in theconverting unit 330, and into a filter strength determining circuit 304functioning as the control-signal output means 320.

[0176] The low-pass filter 303 filters the 10-bit expanded image signalS12 to output a signal S13. This output signal S13 of the low-passfilter 303 is similar to the above-explained signal S3 of FIG. 6, and isentered into the multiplier 305.

[0177] In the multiplier 305, an output signal S15 obtained bymultiplying the signal S13 by a coefficient “a” is sent to themultiplier 307, namely

S15=S13×a,

[0178] note 0≦a≦1.

[0179] This coefficient “a” is controlled by the filter strengthdetermining circuit 304.

[0180] In the multiplier 306, another output signal S14 obtained bymultiplying the output signal S12 from the 10-bit expanding circuit 302by another coefficient (1−a) is supplied to the adder 307, namely:

S14=S12×(1−a).

[0181] This coefficient (1−a) is controlled by the filter strengthdetermining circuit 304.

[0182] The adder 307 adds the signal S14 with the signal S15 to obtainan output signal S16 which will then be supplied to an output terminal308, namely: $\begin{matrix}\begin{matrix}{{S16} = {{S14} + {S15}}} \\{= {{{S12} \times \left( {1 - a} \right)} + {{S13} \times {a.}}}}\end{matrix} & (1)\end{matrix}$

[0183] Subsequently, the filter strength determining circuit 304 willnow be explained.

[0184] In the filter strength determining circuit 304 shown in FIG. 21,the output from the low-pass filter 303 is added with the output fromthe 10-bit expanding circuit 302 at a preselected ratio, and the addedoutput is derived. This is because the high frequency component of theoutput signal from the low-pass filter 303 has been lost. Therefore, theoriginal image is added to this filter output at a preselected ratio inorder to recover this high frequency component. From the above-describedformula (1), when a=0, the original image is outputted. When a=1, thefilter output is directly outputted. The coefficient “a” may becomearbitrary values from 0 to 1. When this coefficient “a” is approximatedto “0”, the filter output is close to the original image. When thiscoefficient “a” is approximated to “1”, the filter output is emphasized.

[0185] The filter strength determining circuit 304 determines thecoefficient “a” contained in the formula (1), namely the ratio of thesignal S12 to the signal S13.

[0186] The method for determining this multiply coefficient “a” will nowbe explained.

[0187] A great effect achieved by bit-expanding the input image so as tosmooth it appears in a flat portion. This is because a visualcharacteristic of a human is highly sensitive to a flat portion. As aresult, the flatness degree of the image signal is measured by thefilter strength determining circuit 304, and then the coefficient “a”may be determined based on this flatness degree.

[0188] A flatness degree of an image signal implies, for example,dispersion of an image. When the dispersion of the image becomes small,this image becomes flat. In this case, the filter strength determiningcircuit 304 may be so arranged as to measure dispersion while a certainpixel is recognized as a center. The filter strength determining circuit304 determines the coefficient “a” as indicated in, for instance, FIG.22 in accordance with the measured dispersion, and then outputs thiscoefficient “a” to the multiplier 306 and another coefficient (1−a) tothe multiplier 305. The coefficient “a” is determined in unit of pixel,and is properly processed.

[0189] As another parameter indicating a flatness degree of an imageunder measurement in the filter strength determining circuit 304, adynamic range of this image may be used. A dynamic rage of an imageimplies a difference between a maximum value and a minimum value of animage signal. Since a flat background is a plain surface, a dynamicrange thereof is narrow. Since a human and an object own curvatures,dynamic ranges thereof become wide. As a consequence, the smaller thedifference between the maximum value and the minimum value of the imagesignal becomes, the more the image becomes flat. In this case, thefilter strength determining circuit 304 may be so arranged as to measurea dynamic range while positioning a certain pixel as a center, namely:dynamic range=maximum value−minimum value. Depending upon the measureddynamic range, the coefficient “a” is determined as illustrated in FIG.23, and then is outputted.

[0190] As a modification of this filter strength determined circuit 304,the coefficient “a” may be determined in response to a luminance signallevel of an image signal. This may introduce a great effect of the10-bit expansion by such a fact that the darker brightness of a portion,the easier the 1 bit difference can be recognized in the 8-bit image. Inthis case, the filter strength determining circuit 304 determines thecoefficient “a” in response to the luminance signal level, asrepresented in FIG. 24, and outputs the determined coefficient “a”.

[0191] Alternatively, a plurality of these coefficient determiningmethods may be employed at the same time, or other coefficientdetermining methods may be utilized.

[0192] Next, a description will now be made of the noise eliminatingfilter 300 functioning as the noise eliminating means shown in FIG. 21.

[0193] This noise eliminating filter 300 may be used or may not be used.The following description is made in such a case that the noiseeliminating filter 300 is inserted/connected to the front stage of the10-bit expanding circuit 302 equal to the bit expanding means.

[0194] This noise eliminating filter 300 eliminates a noise componentcontained in the image signal, and is realized by, for instance, alow-pass filter. As this low-pass filter, a linear filter or a nonlinearfilter may be employed.

[0195]FIG. 25A shows an image signal SS1 containing a large number ofnoises. In FIG. 25B, the output signal of the noise eliminating filter300 when this image signal SS1 is inputted is indicated as “SS2”. Whenthe noise eliminating filter 300 is used, the noises are removed and theimage is smoothed, so that there are manu flat portions. As aconsequence, there is another problem that a pseudo contour may beproduced.

[0196] According to the embodiment of the present invention, a step of a1 bit difference in 8 bits is smoothed at a flat portion of an imagesignal by making up an intermediate gradation. However, if the noise iscontained in this image, then the flat portion cannot be discriminatedfrom the step.

[0197] Under such a reason, the noise eliminating filter 300 isinserted/connected to the front stage of the 10-bit expanding circuit302 functioning as the bit expanding means according to this embodiment,and then the 8-bit-to-10-bit converting process is carried out afterperforming the noise elimination by the noise eliminating filteringprocess. As a consequence, the above-described two problems can besolved.

[0198] A similar effect may be apparently achieved byinserting/connecting such a noise eliminating filter 300 to the frontstage of the 10-bit expanding circuit 102 functioning-as the bitexpanding means of the embodiment of FIG. 6.

[0199] It should be understood that the present invention is not limitedto the above-described embodiments, but may be modified, substituted,and changed. For example, the bit conversion from 8 bits to 10 bits maybe alternatively substituted by converting n bits into m bits, wheresymbols “n” and “m” are integers, and have a relationship of n<m. Also,the coding/decoding methods as explained in FIG. 12 and FIG. 20 may besubstituted by arbitrary coding/decoding methods.

[0200] According to the present invention, the n-bit quantized inputimage signal is bit-expanded into the m-bit image signal in response tothe control signal. Accordingly, the intermediate gradation is producedto mitigate the quantizing noise.

[0201] Also, since the input image signal is smoothed by the low-passfilter, and at the same time, such a process operation is carried out inorder not to lose the high frequency component of the input imagesignal, the pseudo contours caused by the capability limit of the imagesignal does not appear in emphasized manner without deteriorating theresolution.

What is claimed is:
 1. An image signal processing apparatus comprising:bit expanding means for bit-expanding an n-bit quantized inputimage-signal into an m-bit (symbols “n” and “m” are integers, and own arelationship of n<m); control signal output means for outputting acontrol signal based upon said input image signal; and a converting unitfor adaptively converting the signal from said bit expanding means intoan m-bit signal in response to the control signal from said controlsignal output means.
 2. An image signal processing apparatus as claimedin claim 1 wherein: said converting unit produces a converting outputsignal to said m-bit by using an output signal derived from a low-passfilter.
 3. An image signal processing apparatus as claimed in claim 2wherein: said converting means adds a high frequency component of saidinput image signal to the output signal derived from said low-passfilter so as to produce said converting output signal to said m-bit. 4.An image signal processing apparatus as claimed in claim 2 wherein: saidconverting unit produces said converting output signal to the m-bit byemploying said n-bit of the input image signal and lower order (m−n)bits of the output signal derived from said low-pass filter.
 5. An imagesignal processing apparatus as claimed in claim 2 wherein: said controlsignal output means adds the output signal from the low-pass filter withsaid input image signal at a preselected ratio.
 6. An image signalprocessing apparatus as claimed in claim 5 wherein: said predeterminedratio is determined based on at least one of dispersion, a dynamicrange, and a luminance level of the image signal.
 7. An image signalprocessing apparatus as claimed in claim 1 wherein: noise eliminatingmeans is provided at a front stage of said bit expanding means.
 8. Animage signal processing method comprising the steps of: bit-expanding ann-bit quantized input image signal to an m-bit (symbol “n” and “m” areintegers and owns a relationship of n>m); and adaptively producing lowerorder (m−n) bits in accordance with an image nature of said input imagesignal to thereby be converted into an m-bit signal.
 9. An image signaldecoding apparatus for decoding an image signal coded by predictionimage coding, comprising: bit expanding means for bit-expanding such ann-bit digital image signal before being decoded and A/D-converted toproduce an m-bit (symbols “n” and “m” are integers and own arelationship of n>m); control signal output means for outputting acontrol signal in response to said input image signal; and a convertingunit for adaptively converting the signal from said bit expanding meansinto an m-bit signal in response to the control signal derived from saidcontrol signal output means.